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METHOD FOR DEBLOCKING AND TRANSCODING A MEDIA STREAM 

TECHNICAL FIELD 

Embodiments of the present invention relate to the field of digital media. 
Specifically, embodiments of the present invention relate to a method and system for 
deblocking and transcoding a media stream. 

BACKGROUND ART 

One of the important video transcoding applications is bit rate reduction 
transcoding. Rate reduction transcoding is typically achieved using either truncation 
based transcoding or by requantization-based transcoding. Selective coefficient 
truncating transcoding typically achieves a relatively small amount of bit rate 
reduction and is rarely used often in typical video transcoding applications. 

Requantization-based transcoding achieves rate reduction by requantization 
using coarser quantization steps. Since the transform-domain coefficients are coarsely 
quantized, blocking artifacts may be produced. In particular, blocking artifacts arise 
as a result of each block being compressed independently. Blocking artifacts may 
appear as edges or produce ringing between blocks. 

Numerous methods have been developed to reduce the blocking artifacts for 
decoded pixel blocks. Specifically, current deblocking applications typically decode 
the pixel blocks before deblocking and re-encode the pixel blocks after decoding. 
However, for transcoding applications, the decoding and re-encoding cycle may result 
in increased blocking artifacts, as the decoding and re-encoding may rely on 
information that is not be available. Furthermore, due to the higher compression 
provided by transcoding applications, blocking artifacts get worse as the compression 
is increased. For example, requantization-based transcoding applies the 
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requantization on the transform-domain coefficients so that a costly inverse 
transformation is avoided. Currently, deblocking cannot be performed in conjunction 
with transcoding in the transform domain. 
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DISCLOSURE OF THE INVENTION 

Various embodiments of the present invention, a method for deblocking and 
transcoding a media stream, are described. A coefficient associated with a block of 
pixels of the media stream is received. A deblocking operation is performed on the 
5 coefficient to generate a second coefficient. Quantization is performed on the second 

coefficient to generate a transcoded coefficient. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporated in and form a part of this 
specification, illustrate embodiments of the invention and, together with the 
description, serve to explain the principles of the invention: 

FIGURE 1 illustrates a block diagram of transcoding system in accordance 
with an embodiment of the present invention. 

FIGURE 2 illustrates a graph of exemplary quantization bins for two 
quantizers in accordance with an embodiment of the present invention. 

FIGURE 3 illustrates a flow chart of a process for determining responsiveness 
of a coefficient of a media stream in accordance with an embodiment of the present 
invention. 

FIGURE 4 illustrates a flow chart of a process for determining responsiveness 
of a coefficient of a media stream using a uniform scalar quantizer with rounding to 
nearest in accordance with an embodiment of the present invention. 

FIGURE 5A illustrates a graph of an exemplary distribution of responsive and 
unresponsive coefficients using a uniform scalar quantizer with rounding to nearest in 
accordance with an embodiment of the present invention. 

FIGURE 5B illustrates a graph of an exemplary distribution of responsive and 
unresponsive coefficients using a uniform scalar quantizer with rounding down in 
accordance with an embodiment of the present invention. 
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FIGURE 6 illustrates a block diagram of transcoding system for deblocking 
responsive coefficients in accordance with an embodiment of the present invention. 

FIGURE 7 illustrates an exemplary quantization error table in accordance with 
an embodiment of the present invention. 

FIGURE 8 illustrates a flow chart of a process for determining an optimal 
quantization step size in accordance with an embodiment of the present invention. 
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BEST MODE FOR CARRYING OUT THE INVENTION 

Reference will now be made in detail to various embodiments of the invention, 
examples of which are illustrated in the accompanying drawings. While the invention 
will be described in conjunction with these embodiments, it will be understood that 
5 they are not intended to limit the invention to these embodiments. On the contrary, 

the invention is intended to cover alternatives, modifications and equivalents, which 
may be included within the spirit and scope of the invention as defined by the 
appended claims. Furthermore, in the following description of the present invention, 
numerous specific details are set forth in order to provide a thorough understanding of 
10 the present invention. In other instances, well-known methods, procedures, 

components, and circuits have not been described in detail as not to unnecessarily 
obscure aspects of the present invention. 

Aspects of the present invention may be implemented in a computer system 
15 that includes, in general, a processor for processing information and instructions, 

random access (volatile) memory (RAM) for storing information and instructions, 
read-only (non- volatile) memory (ROM) for storing static information and 
instructions, a data storage device such as a magnetic or optical disk and disk drive for 
storing information and instructions, an optional user output device such as a display 
20 device (e.g., a monitor) for displaying information to the computer user, an optional 

user input device including alphanumeric and function keys (e.g., a keyboard) for 
communicating information and command selections to the processor, and an optional 
user input device such as a cursor control device (e.g., a mouse) for communicating 
user input information and command selections to the processor. 

25 

Method and System for Deblocking and Transcoding a Media Stream 
Figure 1 illustrates a block diagram of transcoding system 100 in accordance 
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with an embodiment of the present invention. Transcoding system 100 utilizes 
requantization for bit rate reduction transcoding and has deblocking capability. 
Transcoding system 100 comprises three modules: inverse quantizer 105, deblocking 
mechanism 1 10, and transcoder 115. It should be appreciated that transcoding system 
100 may be implemented within a computer system. 

Transcoding system 100 receives a coefficient associated with a block of pixels 
of a media stream. Embodiments of the present invention pertain to media stream for 
transmitting graphical data, such as video streams. For brevity and clarity, the term 
video stream is used in the present application. In one embodiment, the coefficient is 
a quantized coefficient C q . In one embodiment, quantized coefficient C q is received 
from a compressed video stream. In one embodiment, quantized coefficient C q is a 
discrete cosine transform (DCT) coefficient. 

Inverse quantizer 105 is operable to perform an inverse quantization operation 
(Qr 1 ) on quantized coefficient C q using quantization step size s\. Inverse quantizer 
105 outputs dequantized coefficient C. It should be appreciated that the accuracy of 
dequantized coefficient C may not be known because information of the original 
coefficient may be unavailable. Since it is a lossy reconstruction, dequantized 
coefficient C may bring blocking artifacts to the reconstructed picture. It should also 
be appreciated that embodiments of the invention may not require inverse quantizer 
105 where the coefficient received at transcoding system 100 is already dequantized. 

Deblocking mechanism 1 10 is operable to perform a deblocking operation on 
dequantized coefficient C. In one embodiment, a DCT domain deblocking operation is 
performed to obtain a second coefficient C\ DCT domain deblocking operations 
modify DCT coefficients depending on the neighboring coefficients in the subbands. 
It should be appreciated that DCT domain deblocking operations are understood in 
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the art. An example of a DCT domain deblocking operation is described in 
Triantafyllidis et al., "Blocking Artifact Reduction in Frequency Domain," Proc. 
ICIP2001, pp 269-272, 2001, which is incorporated herein by reference as 
background material. 

Transcoder 1 15 is operable to requantize second coefficient C using 
quantization step size s 2 , resulting in transcoded coefficient C q ' (e.g., a re-quantized 
coefficient). In one embodiment, quantization step size s 2 is produced by a rate 
control module of transcoder 115 that regulates the bit budget spent for each 
macroblock depending on a block complexity measure and current bit-consumption 
status. In one embodiment, quantization step size s 2 is less than quantization step 
size Si. Therefore, transcoded coefficient C q ' is less than or equal to quantized 
coefficient C q , possibly requiring a fewer number of bits to code transcoded 
coefficient C q \ Bit rate reduction is thus achieved. 

It should be understood that in video transcoding, coefficients modified after a 
deblocking operation are still subject to transcoding (e.g., re-quantization) that may 
render the deblocking modification ineffective. In particular, some coefficients may be 
responsive to a deblocking operation while other coefficients may be unresponsive to 
a deblocking operation. 

Embodiments of the invention are directed towards determining whether a 
coefficient is responsive to deblocking in order to accelerate the deblocking operation 
in transcoding. Figure 2 is an example illustrating that some coefficients may be 
responsive to a deblocking operation while other coefficients may be unresponsive to 
a deblocking operation. Specifically, Figure 2 illustrates a graph 200 of exemplary 
quantization bins for two quantizers in accordance with an embodiment of the present 
invention. In one embodiment, quantizer Qj and quantizer Q 2 are both uniform scalar 
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quantizers, Graph 200 illustrates two different sets of bins generated with two 
different quantization steps. The original quantization step size for quantizer Qi is 
quantization step size Si and the transcoding step size for quantizer Q 2 is 
quantization step size s 2 . 

5 

For illustrative purposes, if an input coefficient falls into bin a of quantizer 
Q lt the reconstructed coefficient is represented by the middle value of bin a, as 
represented by the arrow. After deblocking, the adjusted coefficient can vary within 
the range of bin a. Consequently, the requantization result can fall either in bin A or 

10 bin B of quantizer Q 2 . If a post-deblocking coefficient may end up in different bins of 

Q 2 , the coefficient is responsive to deblocking. Alternatively, if an input coefficient 
falls into bin a ', the post-deblocking coefficient always falls into bin C regardless of 
how the deblocking process alters a 9 since the whole range of a ' is within C. The 
input coefficient is unresponsive to deblocking. Therefore, for unresponsive 

15 coefficients, no deblocking operation need be applied on them, as deblocking is 

ineffective. 

The value of the unresponsive coefficients can be derived as follows given the 
quantizer function f 9 quantization steps Si and s 2 . Assuming a uniform scale quantizer 
20 with rounding to nearest, the inverse quantization function f x is considered. Given an 

input DCT coefficient (a quantized DCT coefficient) C q =n, the reconstructed 
coefficient is C n =f x (C g ,s) = ns, where s is the quantization step size. The 
quantization lower bound of C„ is defined as: 

C L n =ns-[s/2\ (1) 

25 

and the upper bound is defined as: 
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C v n =ns-[sl2\ + s-l 



(2) 



Note that the round-to-nearest quantizing (values at 0.5 are rounded to the larger 
magnitude) leads to the round-down term in Equations (1) and (2). Let i that belongs 
to C q , the set of input coefficients, and j that belongs to C q \ the set of coefficients 
after requantization, C q and C q ' are subsets of integers, from Equations (1) and (2) 
results in: 



Cf = is x -[s x l2\ , C 1 ? =is x -[s x l2\ + s x -\ 
Cj = js 2 - [s 2 12\ , C u j = js 2 - [s 2 /2J + s 2 - 1 



(3) 



The set of unresponsive coefficients is defined as: 



U = j/|V/,j G int, Cf * C) and C v t <; C U j] 



(4) 



Plugging Equation (3) into the condition items in Equation (4) and noting that 
[s/2] + [s/2\ = s, a quantization theorem is derived. Specifically, for a uniform scalar 
quantizer (round-to-nearest quantizing), given first quantizer step size s x and 
requantizer step size s 2 , the magnitude of unresponsive coefficients i that belongs to 
C q , satisfies: 



V s \ <* < {Pi + 



(5) 



and the magnitude of the responsive coefficients / ' fall into the intervals defined by: 



7*2 + 



\/s l <i'<\js 2 + 



\/ Sl 



(6) 
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where j belongs to C q \ 

Alternatively, if round-down instead of round-to-nearest is considered in 
quantizing, the intervals for responsive coefficients become: 



Equation (7) considers a round-down scheme with uniform scalar quantizer only. It 
should be appreciated that other scalar quantizers can be used. For example, a round- 
down scheme with dead zone quantizer (for quantization of inter pictures in MPEG) 
can be evaluated similarly. 

In general, given any (si, s 2 ) pair, a coefficient responsiveness table for 
recording responsive or unresponsive coefficients for a range of coefficients can be 
obtained according to a quantization operation (e.g., quantization function J). For 
example, if the quantized coefficients are obtained from an MPEG-2 video stream, the 
range /?=[0,1024]. It should be appreciated that the range depends on media stream 
encoding, such as MPEG, H.26x, and JPEG. 

In one embodiment, the quantization operation is a uniform scalar quantizer 
with rounding to nearest as shown in Equation (6). In another embodiment, the 
quantization operation is a uniform scalar quantizer with rounding down as shown in 
Equation (7). For ease of explanation, the left side of Equations (6) and (7) is denoted 
as Lj and right side is denoted as Uj. For both Equations (6) and (7), it can be proven 
that Lj - Uj r =1 . In other words, there is at most one responsive coefficient in each 
interval. Therefore, Equations (6) and (7) can be used to generate a coefficient 




(7) 



'2 
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responsiveness table indicating the responsiveness for a coefficient. 

Figure 3 illustrates a flow chart of a process 300 for determining 
responsiveness of a coefficient of a media stream in accordance with an embodiment 
5 of the present invention. In one embodiment, process 300 is carried out by 

processors and electrical components (e.g., a computer system) under the control of 
computer readable and computer executable instructions. Although specific steps are 
disclosed in process 300, such steps are exemplary. That is, the embodiments of the 
present invention are well suited to performing various other steps or variations of the 
1 0 steps recited in Figure 3 . 

At step 310, a plurality of first coefficients and a plurality of second 
coefficients associated with plurality of blocks of pixels, a first quantization step size, 
a second quantization step size, and a quantization operation are received. In one 
15 embodiment, the plurality of first coefficients are quantized coefficients and the 

plurality of second coefficients are transcoded coefficients. In one embodiment, the 
quantization operation is a uniform scalar quantizer with rounding to nearest. In 
another embodiment, the quantization operation is a uniform scalar quantizer with 
rounding down. 

20 

At step 320, the quantization operation is performed for a first coefficient of 
the plurality of first coefficients and a second coefficient of the plurality of second 
coefficients. The quantization operation utilizes the first quantization step size, the 
second quantization step size and the second coefficient. In one embodiment, the 
25 quantization operation determines whether the first coefficient falls within an interval 

defined by the first quantization step size, the second quantization step size and the 
second coefficient. 
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At step 330, it is determined whether the first coefficient is responsive based 
on the quantization operation. In one embodiment, if the coefficient falls within an 
interval as defined by the quantization operation, the first coefficient is identified as 
being responsive. Alternatively, if the coefficient does not fall within the interval as 
defined by the quantization operation, the first coefficient is identified as being 
unresponsive. In one embodiment, the quantization operation defines a lower bound 
and an upper bound. Based on the first quantization step size, the second 
quantization step size and the second coefficient, the lower bound and upper bound 
are determined. If lower bound is less than the first coefficient and the upper bound is 
greater than the first coefficient, the first coefficient is indicated as responsive. 

At step 340, it is determined whether more coefficients require processing by 
the quantization operation. In one embodiment, it is determined whether the 
quantization operation has been performed on every combination of first coefficients 
of the plurality of first coefficients and second coefficients of the plurality of second 
coefficients. If there are more coefficients that require processing by the quantization 
operation, process 300 returns to step 320 with the next combination of first 
coefficients and second coefficients. Alternatively, if there are no more coefficients 
that require processing by the quantization operation, process 300 proceeds to step 
350. 

At step 350, a coefficient responsiveness table is generated comprising a 
distribution of responsiveness of the plurality of first coefficients for the first 
quantization step size. A first axis of the coefficient responsiveness table 
corresponds to the plurality of first coefficients and the second axis corresponds to 
the second quantization step size. It should be appreciated that the coefficient 
responsiveness table can be generated prior to the completion of step 340. That is, 
the coefficient responsiveness table can be generated one coefficient at a time, such 
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that once responsiveness of a first coefficient is determined at step 330, the 
responsiveness is indicated in the coefficient responsiveness table. 

Figure 4 illustrates a flow chart of a process 400 for determining 
responsiveness of a coefficient of a media stream using a uniform scalar quantizer with 
rounding to nearest in accordance with an embodiment of the present invention. In 
one embodiment, process 400 is carried out by processors and electrical components 
(e.g., a computer system) under the control of computer readable and computer 
executable instructions. Although specific steps are disclosed in process 400, such 
steps are exemplary. That is, the embodiments of the present invention are well 
suited to performing various other steps or variations of the steps recited in Figure 4. 

At step 402 of Figure 4, all possible quantized coefficients are set as 
unresponsive. At step 404, Lj (e.g., the right side of Equation (6)) and Uj (e.g., the left 
side of Equation (6)) are determined starting from the first interval where a transcoded 
coefficient of a plurality of transcoded coefficients is set equal to one (e.g.,y=l). At 
step 406, the first possible quantized coefficient of the plurality of quantized 
coefficients is set equal to one (e.g., i=l). 

At step 408, it is determined whether Equation (6) is satisfied for the 
quantized coefficient. In other words, it is determined whether the quantized 
coefficient falls within the interval of Lj and Uj. If the quantized coefficient falls 
within the interval of Lj and Up as shown at step 410, the quantized coefficient is set 
as responsive, and process 400 proceeds to step 412. Alternatively, if the quantized 
coefficient does not fall within the interval of Lj and Uj process 400 proceeds to step 
412. 

At step 412 it is determined whether all values of quantized coefficients have 
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been processed at step 408. If all values of quantized coefficients have not been 
processed, as shown at step 414, the next possible quantized coefficient is selected 
and process 400 proceeds to step 408. Alternatively, if all values of quantized 
coefficients have been processed, process 400 proceeds to step 416. 

At step 416 it is determined whether all intervals of Lj and Uj have been 
processed at step 408. If all intervals have not been processed, as shown at step 418, 
the next possible transcoded coefficient is selected and process 400 proceeds to step 
408. Alternatively, if all intervals have been processed, process 400 proceeds to step 
420. At step 420, process 400 ends. 

Figure 5 A illustrates a graph 500 of an exemplary distribution of responsive 
and unresponsive coefficients using a uniform scalar quantizer with rounding to 
nearest in accordance with an embodiment of the present invention. Graph 500 
shows the actual distribution of responsive and unresponsive coefficients for an 
MPEG-2 video stream with a first quantization step size of three (si=3). The vertical 
axis indicates the second quantization step size s 2 which ranges from 1-31 in the case 
6f MPEG-2 video streams, and the horizontal axis indicates the quantized Coefficient 
C q (e.g., first coefficient), which ranges from 1-1024 in the case of MPEG-2 video 
streams. For simplicity, graph 200 only illustrates the first 30 quantized coefficients. 
As shown, 'o' indicates a responsive coefficient and 'x' indicates an unresponsive 
coefficient. Also, since the second quantization step size is greater than the first 
quantization step size, it should be understood that the responsiveness is only 
determined for second quantization step size from four through thirty-one. In one 
embodiment, graph 500 is generated for each first quantization step size. However, it 
should be appreciated that a three-dimensional table can be generated that includes all 
first quantization step sizes, second quantization step sizes, and quantized 
coefficients. 
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Figure 5B illustrates a graph 510 of an exemplary distribution of responsive 
and unresponsive coefficients using a uniform scalar quantizer with rounding down in 
accordance with an embodiment of the present invention. Graph 5 10 is generated in a 
manner similar to graph 500 of Figure 5 A, and is for an MPEG-2 video stream having 
a first quantization step size of three. Due to the different quantizer used, the 
responsiveness of coefficients is different than the quantizer used at graph 500. 

It should be appreciated that second quantization step size s 2 is typically no 
smaller than first quantization step size S\. Thus, the transcoded coefficient is smaller 
than the input quantized coefficient. In bit rate reduction transcoding, for any given / 
that belongs to C q , after deblocking there are at most two possible requantization 
outputs. Using this information, a deblocking capable transcoder can be implemented 
that only performs a deblocking operation on responsive coefficients, thus facilitating 
efficient throughput of input coefficients. 

Figure 6 illustrates a block diagram of transcoding system 600 for deblocking 
responsive coefficients in accordance with an embodiment of the present invention. 
Transcoding system 600 utilizes requantization for bit rate reduction transcoding and 
has deblocking capability. Transcoding system 600 comprises inverse quantizer 605, 
coefficient responsiveness table 610, responsiveness selector 620, deblocking 
mechanism 610, and transcoder 615. It should be appreciated that transcoding system 
600 may be implemented within a computer system. 

Transcoding system 600 receives a coefficient associated with a block of pixels 
of a media stream. In one embodiment, the coefficient is a quantized coefficient C q . 
In one embodiment, quantized coefficient C q is received from a compressed video 
stream. In one embodiment, quantized coefficient C q is a discrete cosine transform 
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(DCT) coefficient. Inverse quantizer 605 is operable to perform an inverse 
quantization operation (Qf 1 ) on quantized coefficient C q using quantization step size 
S\. Inverse quantizer 605 outputs dequantized coefficient C. In one embodiment, 
inverse quantizer operates in the same manner as inverse quantizer 105 of Figure 1. 

5 

Dequantized coefficient C is transmitted to transcoder 615. Transcoder 615 is 
operable to perform a transcoding (e.g., quantization) operation (Q 2 ) on dequantized 
coefficient C using quantization step size s 2 . Transcoder 615 outputs transcoded 
coefficient C q \ In one embodiment, transcoder 615 operates in the same manner as 
10 transcoder 115 of Figure 1. 

Responsiveness selector 630 performs a table-lookup operation using quantized 
coefficient C q as an input, as well as quantization step sizes S\ and s 2 . In one 
embodiment, the table lookup operation is based on coefficient responsiveness table 
15 620 which is generated beforehand. In one embodiment, coefficient responsiveness 

table 620 is generated as described at process 300 of Figure 3 or process 400 of Figure 
4. Based on quantization step size s 2 proposed by a rate control module, an input 
coefficient i is mapped to a corresponding C q ' of coefficient responsiveness table 620. 

20 Responsiveness selector 630 performs a table-lookup operation and the 

output of 630 is used to control a switch arrow 635. If i that belongs to C q is 
unresponsive according to coefficient responsiveness table 620, switch arrow 635 is 
connected to unresponsive node 640, transcoded coefficient C q ' is output. 
Alternatively, if i is responsive according to coefficient responsiveness table 620, 

25 switch arrow 635 is connected to responsive node 645. Subsequently, a deblocking 

process of deblocking mechanism 610 is applied on the transcoded coefficient C q ' 
according to dequantized coefficient C. Transcoded coefficient C q ' is modified by ±1 
or zero, and is output as C q ". In one embodiment, deblocking mechanism 610 
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operates in the same manner as deblocking mechanism 110 of Figure 1 . In one 
embodiment, a memory unit of transcoding system 600 is used to pass dequantized 
coefficient C to deblocking mechanism 610. 



If quantization step size s x is small, there is typically minimal blocking 
introduced into the video stream. However, if quantization step size s 2 is much 
greater than quantization step size Si, there may be substantial blocking artifacts 
introduced. The distribution of responsive coefficients may become sparse with 
increasing quantization step size s 2 . Noting that the interval between the lower and 
upper bounds in Equation (5) equals s 2 /s\-l 9 the following corollary can be derived. In 
general, for a uniform scalar quantizer, given first quantization step size S\ and second 
quantization step size s 2 , the number of unresponsive coefficients increases when 
s 2 /si increases. This corollary indicates that a deblocking operation has limited impact 
when requantization is much more coarser. In some special cases, no responsive 
coefficients are available at all. Note from the example shown in Figure 5 A that when 
s 2 =9, no coefficients are responsive. 

In general, for a uniform scalar quantizer, given first quantization step size s x 
and second quantization step size s 2 , if 



Method for Determining an Optimal Quantization 
Step Size for Transcoding a Media Stream 




where k = (1, 2, . . .), there are no responsive coefficients. 
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It is therefore important to avoid choosing second quantization step size s 2 as 
in Equation (8) if a deblocking-capable transcoder is desired. From the example 
shown in Figure 5 A, there are more responsive coefficients when second quantization 
5 step size s 2 =8 or 10. Therefore, by altering the selection of second quantization step 

size s 2 , the number of responsive coefficients may increase so that the deblocking can 
be more effective. 

In one embodiment, a bigger or smaller quantization step size in the vicinity of 
10 quantization step size s 2 is selected, leading to possibly more responsive coefficients. 

To select an appropriate quantization step size, a plot of quantization error is 
generated. In one embodiment, the quantization error is plotted based on standard 
TM5 quantization procedure employed in the MPEG standard. Given the range of 
quantization step size s 2 from 1 to 31, and possible magnitudes of DCT coefficients 
15 from 0 to 2048, a quantization error table containing all possible quantization errors 

for each (s 2 , C q ) pair can be generated. 

Figure 7 illustrates an exemplary quantization error table 700 in accordance 
with an embodiment of the present invention. Specifically, quantization error table 

20 700 shows the quantization error for DCT coefficients for values from 0 to 1028 

using the MPEG standard. For simplicity, only the first 64 coefficients are shown. 
The quantizer function and the default intra quantizer matrix used in MPEG are 
considered here. The visualization of the table only shows the results for the (0,1), 
(1,0) and (1,1) coefficients (the value of these entries in the default quantizer matrix is 

25 16). The troughs (darker cells) indicate smaller quantization error. For a vertical line 

(for example the vertical line for coefficient value at 40) in Figure 7, multiple troughs 
are encountered, which indicates that a higher quantization step size s 2 value does not 
necessarily increase the quantization error. On the contrary, if quantization step size 
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s 2 is selected carefully, the quantization error may be maintained or even reduced. 

The quantization property described above proves valuable for the 
deblocking-capable transcoder. An optimal quantization step size s 2 can be selected 
to boost the number of responsive coefficients so that the deblocking mechanism (e.g., 
deblocking mechanism 110 of Figure 1 or deblocking mechanism 610 of Figure 6) can 
be more effective. Furthermore, the altered quantization step size maintains the bit 
rate and the quality of the coded pictures. Based on this, a deblocking-capable 
transcoder can be implemented with enhanced deblocking capability. The enhanced 
deblocking capability can be implemented in transcoding system 100 of Figure 1 and 
transcoding system 600 of Figure 6 by selecting an optimal second quantization step 
size s 2 . 

Figure 8 illustrates a flow chart of a process 800 for determining an optimal 
quantization step size in accordance with an embodiment of the present invention. In 
one embodiment, process 800 is carried out by processors and electrical components 
(e.g., a computer system) under the control of computer readable and computer 
executable instructions. Although specific steps are disclosed in process 800, such 
steps are exemplary. That is, the embodiments of the present invention are well 
suited to performing various other steps or variations of the steps recited in Figure 8. 

At step 810, an input quantization step size and a plurality of coefficients of a 
macroblock are received. In one embodiment, an input quantization step size and a 
plurality of coefficients are received for a group of blocks (e.g., in the case of JPEG 
encoding). 

At step 820, a magnitude distribution of non-zero coefficients of the plurality of 
coefficients is determined. 



20 



200309817-1 



At step 830, a plurality of candidate quantization step sizes is determined 
based on a first table. In one embodiment, the first table is a quantization error table. 

At step 840, a quantization error for a range of quantization step sizes 
including the input quantization step size is determined. In one embodiment, the 
range of quantization step sizes comprises all quantization step sizes between the 
input quantization step size plus three and the input quantization step size minus 
three. In other words, for quantization step size s 2 , the range of quantization step 
sizes ranges from s 2 -3 to s 2 +3 

At step 850, a plurality of candidate quantization step sizes from is selected 
form the range of quantization step sizes. In one embodiment, the plurality of 
candidate quantization step sizes produce a quantization error not greater than 
quantization error associated with the input quantization step size. 

At step 860, the optimal quantization step size is determined from said 
plurality of quantization step sizes based on a second table. In one embodiment, the 
second table is a coefficient responsiveness table. In one embodiment, the optimal 
quantization step size is selected as a quantization step size of plurality of 
quantization step sizes that has the highest number of responsive coefficients 
according to the coefficient responsiveness table. 

It should be appreciated that in the case of MPEG, one quantization step size 
s 2 is applied to the quantization of 6 to 12 blocks in a macroblock. The distribution 
of non-zero coefficients is obtained considering all the blocks. Weighting factors can 
be assigned to coefficients at different frequencies based on their importance to the 
reconstruction quality. In one embodiment, using the lower frequency coefficients 
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(e.g., coefficients in the upper left 2x2 in a block) can be implemented. 

Various embodiments of the present invention provide an analytical model for 
identifying transform coefficients that are responsive to deblocking in a video 
transcoding process. A method that enables a transcoder to avoid unnecessary 
deblocking operations in the transcoding process is provided. An optimal quantizer 
selection method is also provided to increase the number of responsive coefficients so 
that the deblocking capability of the transcoder is enhanced. 

Embodiments of the present invention, a method for deblocking and 
transcoding a media stream, are thus described. While the present invention has been 
described in particular embodiments, it should be appreciated that the present 
invention should not be construed as limited by such embodiments, but rather 
construed according to the following claims. 
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